1 INTERACTIVE SURVEY AND 

2 DATA MANAGEMENT METHOD AND APPARATUS 

3 This application claims the benefit of U. S . Provisional Application 60/208 ,3 64 

4 filed May 3 1 , 2000. This application is also related to co-pending application 

5 INTEGRATED COMMUNICATION SYSTEM AND METHOD, Nelson et al., 

6 incorporated herein by reference. 

7 FIELD OF INVENTION 

The present invention relates to collecting and managing customer satisfaction 

il survey information that enables a product or service provider to evahiate the quaUty of 

is goods and services based on survey ratings and reports from customers. 

Jj BACKGROUND OF THE INVENTION 

p Conventional approaches to access customer satisfaction information generally 

III use standard survey forms or questionnaires by mail or telephone. A survey 

participant may be mailed a standard survey form to complete and return by mail or an 

1 5 agent of the provider may call a participant so that the survey questions may be 

1 6 answered over the telephone. 

17 However, these methods of performing surveys are inaccurate and inefficient, 

1 8 often taking considerable time to collect and process the information. For example, a 

1 9 traditional direct mail survey may take more than 1 00 days before a provider reviews 

20 a final report and the survey participants may not even be recent users of the products 

21 or services. A direct mail process normally determines a random survey sample from 

22 a customer data base provided by disk from the product or service provider. An initial 

23 seven to fourteen days may be allowed for sending a pre-notification letter and a direct 
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1 mailing of the survey to each survey participant. Then, reminder cards to complete 

2 the survey are sent to non-respondents after ten days with a second direct mailing 

3 being sent after an additional thirty days. Once the direct mail effort is completed, 

4 telephone interviews may also be performed to meet the response rate quota. A final 

5 report is only sent to the product or service provider two to three weeks after attaining 

6 the required response rate quota. 

7 Furthermore, although computers are likely to be used to analyze the survey 
4 data, human agents are still typically used to enter the data into a computer or to 

J; perform the actual survey questioning in both traditional mail and telephone surveys. 

f# Unfortunately, human agents are expensive to hire, increasing survey costs, and 

l|f humans often make mistakes, leading to survey inaccuracies. 
1^ One possible solution involves using computer-operated surveys where the 

|g participant uses the touch-tone feature of a telephone to enter replies to questions. 

Q Such a system, however, seems impersonal to the participant, often leads to mistakes 

15 if the participant forgets the key/answer mapping, and is limited to the use of touch- 

16 tone telephones. Such a solution also does not provide an adequate means to obtain 

17 actual participant verbal responses to questions. Actual recorded verbal responses 

18 have the advantage of capturing the participant's subjective tone and emotional state, 

19 as well as capturing the participant's objective opinion. 

20 It would be advantageous to automate the survey process to reduce the number 

21 of human beings utilized in the survey process, to increase the accuracy, reduce the 

22 costs, improve the efficiencies, and overcome the shortcomings of current techniques 

23 identified above. Modem computer and networking technology provides potential 
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1 solutions to these problems. Advances in voice recognition, database design, 

2 computer processing, and computer networking all provide means to improve the 

3 process of performing a survey. 

4 In addition, because all survey participants tend to be asked the same pre- 

5 determined list of questions, little insight into the particular reasons behind the 

6 answers are typically available. Conventional means of solving this problem may 

7 involve having a longer, more complex survey or having the participant state a reason 
for his or her answer. However, the provider of a survey assessment may only be 

m interested in detailed reasons behind an answer in particular situations, such as when a 

f# participant is very dissatisfied with a service or product, while the same provider may 

^ not care why a stirvey participant was satisfied with a different service or product. A 
longer complex survey causes a participant to become frustrated at the relevancy and 

ijl time commitment involved in completing the survey, thus, encouraging the participant 

III not to complete the survey fully or to end the survey prematurely. Likewise, 

15 processing stated "reasons" using standard techniques can be difficult, time- 

16 consuming, and costly. 

17 Conventional written surveys attempt to solve this process, for example, by 

18 telling a participant to "skip section X if you answered 'no' to question Y" or to 

19 "complete section Q only if you answered 'poor' to question T". Telephone or 

20 surveys often require the survey agent to follow similar instructions. This can be 

21 confusing and fi:xistrating to participants and agents alike, otten leading to errors or 

22 incomplete surveys. More sophisticated, automated survey techniques that depend on 
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1 neither the participant nor the agent's understanding of the survey structure would be 

2 advantageous. 

3 Accordingly, an adaptable survey questioning procedure would be useful, one 

4 that determines the depth of questioning on certain topics depending on the answers 

5 given to questions on that topic, and that utilizes computer technology to process 

6 verbal replies. Better utilization of modem computers and software and the reduction 

7 in the reliance of human agents on performing the survey questioning would be 
beneficial to the product and service providers and the survey participants, as well as 

gi any agents performing the surveys. 

# BRIEF SUMMARY OF THE INVENTION 

1 pf According to the present invention, an interview system includes a 

questioning series of core item questions and a second questioning series of drill- 

ig down questions that are provided only when a previous response is within a 

M predetermined range to enable a more focused survey. The second questioning series 

15 may also contain open-ended questions called verbatims where the survey participant 

16 may give a detailed response to further explain an answer. 

17 The invention provides a system of collecting survey information. The system 

18 includes a first questioning series of core item questions developed to support a 

19 survey purpose. A second questioning series of drill-down questions is provided 

20 wherein each of the drill-down questions is associated with one of the core item 

21 questions and one of the drill-down questions is presented only when a response to the 

22 associated core item question meets a specified criterion. 
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1 The system is an automated system. The second questioning series contains at 

2 least one verbatim question presented only when a response to a previous question 

3 meets a specified criterion and wherein a detailed response to the verbatim question 

4 may be given to further explain the response to the previous question. The system 

5 includes a device for recording a response to a verbatim question literally. 

6 The invention also provides a method of collecting and managing survey data. 

7 The steps include presenting a first core item question and recording a response to the 
a,^ first core item question; interpreting the first core item response; presenting at least 
IS one drill-down question and recording a response to the drill-down question only if 

1#== the first core item response meets specified criterion; and presenting a second core 

1 12 item question and recording a response to the second core item question. 

Additional steps include interpreting the response to the drill-down question; 

laj and presenting at least one verbatim and recording a verbatim response if the drill- 

Sp down response is not within the predetermined acceptable response range. Other 

15 steps include presenting a second drill-down question and recording a response to the 

16 second drill-down question only if the response to the one drill-down question meets a 

17 specified criterion; and presenting a second core item question and recording a 

1 8 response to the second core item question. 

19 Further steps include presenting at least one unrelated verbatim and recording 

20 an unrelated verbatim response, wherein the unrelated verbatim does not depend upon 

21 a previous response. 
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1 BRIEF DESCRIPTION OF THE DRAWINGS 

2 FIG. 1 is a block diagram showing the technology platform upon which an 

3 embodiment of the present invention operates; 

4 FIG. 2 is a flow chart describing the survey process with a survey participant; 

5 FIG. 3 is a flow chart illustrating an example of an embodiment of the present 

6 invention; 

7 FIG. 4 is a block diagram showing an survey with responses that did not need 
8_ greater detail; 

%i FIG. 5 is a block diagram illustrating a survey which necessitated drill-down 

questions and verbatims for greater detail. 

1 N DETAILED DESCRIPTION OF THE INVENTION 

The interactive survey and data management method and apparatus collects 

13ij and manages information on a technology platform that fully integrates data 

IG collection, data computation, and data presentation into one system. Unlike 

15 traditional data collection processes, the interactive survey-based interview is 

16 generally only a six to twenty-five day cycle. 

17 As shown in FIG. 1, the major components of the communication system 

1 8 involved with the interactive survey-based interview include a customer database 12, 

19 a local area network 14, a connection device 16, a computer- assisted interview agent 

20 1 8, an interactive voice recognition program with speech recognition 20, a network 

21 server 22, and a presentation software or unit 24 that can be accessed by a computer 

22 by the internet or other private network. 
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1 The customer database 12 contains current uses of the provider's product or 

2 services and is generally updated on a daily or weekly basis. The local area network 

3 14 provides automated database interchange between a provider's computer and a 

4 main system computer to collect survey information such as demographic profiles of 
,5 members of a survey sample, costs, or clinical and biological outcome data. The 

6 communication device 16, a dialer when a telephone interface is used, is an automated 

7 outgoing call device that dials a programmed series of participant telephone numbers 
a=, at scheduled times and transfers a call to an agent 18 when a participant answers. 

9j;j The agent 1 8 is a trained call center agent assisted by a computer telephony 

lik^ interface (not shown) which creates a "call record" that follows the routing of each 

1 i]f outgoing and incoming call and provides instantaneous "pop screens" to agents for 
fielding surveys. As shown in FIG. 2, the agent uses a computer terminal to a start 

l|,y survey by introducing the participant to a survey before transferring the participant to 

lf!3 the interactive voice recognition unit 20 to complete an automated survey. Only oral 

15 responses are required by the participant because speech recognition technology is 

16 used. Thus, there are no touch-tone functions which may confuse or frustrate the 

17 participant. Furthermore, the speech recognition technology provides an adequate 

1 8 means to obtain actual participant verbal responses to questions when necessary. If 

19 the participant prefers a human agent, the agent follows the same question pattern as 

20 the one that is programmed for the automated survey. The participant may also 

21 transfer back to an agent at anytime during an automated survey to ask the agent a 

22 question or to complete the survey with the agent. The agent might intervene based 
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1 on a notification triggered by the participant's responses, lack of response, or some 

2 other criterion. 

3 The network server 22 provides the product or service provider with remote 

4 access to data collected within twenty-four hours of its collection across a virtual 

5 private network, which keeps all data off the public internet and ensures privacy and 

6 security. This server authenticates the end user for access and encrypts and de- 

7 encrypts data transferred to the survey presentation imit 24 on which the outcome 
measurements presentation operates and where all archival and dynamic data is 

^ stored. The communication system is more fully discussed in co-pending application 

l#=i INTEGRATED COMMUNICATION SYSTEM AND METHOD, Nelson et al., 

1 i^f incorporated herein by reference. 

1%. An interactive survey-based interview uses a random sample for the survey 

l|j that may be either a daily or weekly sample from users of the product or service. The 

lH interviews usually begin within 72 hours of the service and approximately five 

15 attempts per sample member are made to conduct an interview during the next six to 

16 twenty- five days. Depending on the type of survey information required, the 

17 interview process cycle may then repeated using a new random sample. Finally, the 

18 data collected each day is available within twenty-four hours by the network server 

19 and may be viewed at the provider's computer terminal by the Internet. 

20 An interactive survey is tailored or customized according to the actual 

21 responses of the participant. As shown in FIG. 3, a flow chart illustrates an example 

22 of an interactive survey, where drill-down questions and open-ended questions called 

23 "verbatims" are deployed by the survey only when the response meets specified 
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1 criteria, for example, when the response is not within a predetermined satisfactory 

2 range. Such a survey may be use with the communication system describe previously 

3 or with other electronic and computer-related devices including hand-held devices and 

4 the Ihtemet. Verbatims are questions for which a response selecting from multiple 

5 choice options is not desired or expected to be appropriate. The verbal response of the 

6 participant to a verbatim question is recorded literally, for example, by a digital audio 

7 recording to be played back later. 

This survey format enables the survey to go into more detailed questioning 

ffl only when a certain response is given by the survey participant. If the response is 

lt=ii within the satisfactory range of answers and no additional information is necessary, 

1 1 ;f the survey proceeds to the next standard survey question. However, if the response is 

l^z not within the satisfactory range of answers, only then is a drill-down question asked 

la j of the survey participant. Furthermore, if a drill-down question response is not within 

ill a satisfactory range, the survey may then proceed to ask for a verbatim response to 

15 explain a previous response in even greater detail before asking the next standard 

16 survey question. Drill-down questioning may include one question or a series of 

17 questions depending on the detail needed. Likewise, one or more verbatims may be 

18 asked of a participant depending if additional questioning on a specific topic is 

19 necessary. This flexibility allows a survey to focus on positive or negative responses 

20 and to mix qualitative and quantitative questions. Thus, the survey gathers greater 

21 detail information only when necessary, enabling the survey to be as short as possible 

22 for each survey participant. 
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1 Referring to FIGS. 4 and 5, an example of an interactive survey is presented 

2 using a simulated eight-question standard survey for health plans. The standard 

3 survey asked of plan participants are divided into five core items 32 or areas of 

4 measurement. Each question 34 is linked to one of the five core items 32. Focus or 

5 mini surveys 36 are also developed for each core item where a provider would want 

6 more details to a response, for example, to determine what type of performance 

7 deficiency has occurred within the core item. These focus siirveys contain the drill- 
8^^^,, down questions and verbatims. In FIG. 4, the focus survey were not necessary 

9^ because all the measiurement ratings 38 by the survey participant are within the 

10|=ii. predetermined acceptable response range of high or moderate satisfaction level. 
1 1'' J In FIG. 5, another survey participant gave low measurement rating to 

1^,, Questions 4 and 8. Using the interactive survey approach, focus or mini surveys are 

1^1 then deployed for these questions that receive a low recommendation. Question 4 

iCi asks five drill-down questions and one verbatim; Question 8 asks four drill-down 

15 questions and two verbatims. Thus, the drill-down questions extend the standard 

16 survey to measure in more detail the response of the previous question. From these 

17 focus surveys, the health provider would be able to learn in detail the cause of low 

18 ratings in response to Questions 4 and 8. With this detailed information, the provider 

19 would be able to develop a quality and performance improvement plan to identify 

20 corrective actions and process changes. 

21 As a result of ongoing quality improvement efforts, the performance of the 

22 health provider should improve and subsequent surveys should result in higher rating. 
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As improvements are made, the focus sxirveys can be changed or redesign to measure 
this change or be suspended for certain core items. 

While the invention has been described with reference to a specific 
embodiment, various changes maybe made and equivalents may be substituted for 
elements thereof by those skilled in the art without departing from the scope of the 
invention, hi addition, other modifications may be made to adapt a particular 
situation or method to the teachings of the invention without departing from the 
essential scope thereof The present invention herein is not to be construed as being 
limited, except insofar as indicated in the appended claims. 
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